Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

added queue functionality #690

Open
wants to merge 8 commits into
base: main
Choose a base branch
from
Open

Conversation

kriyanshii
Copy link
Contributor

@kriyanshii kriyanshii commented Oct 6, 2024

I have implemented queue functionality here. and now it works effortlessly despite having multiple dags. We had multiple hiccups
as it was difficult to find the number of running dags and to maintian queue. for that we have introduced new files. $DAGU_HOME/queue/queue.json(to maintain queue), $DAGU/stats/stats.json(which only stores no of running dags - any dag while running will increment this and while finished running it will decrement this pointer).
Also introdused a new flag "w" as if the dag is coming from the queue, it will directly go to the running stage.
Introdused a new config variable - DAGQueueLength.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant